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Abstract 

Background: Multiple treatment comparison (MTC) meta-analyses are commonly modeled in a Bayesian 
framework, and weakly informative priors are typically preferred to mirror familiar data driven frequentist 
approaches. Random-effects MTCs have commonly modeled heterogeneity under the assumption that the 
between-trial variance for all involved treatment comparisons are equal (i.e., the 'common variance' assumption). 
This approach 'borrows strength' for heterogeneity estimation across treatment comparisons, and thus, ads valuable 
precision when data is sparse. The homogeneous variance assumption, however, is unrealistic and can severely bias 
variance estimates. Consequently 95% credible intervals may not retain nominal coverage, and treatment rank 
probabilities may become distorted. Relaxing the homogeneous variance assumption may be equally problematic 
due to reduced precision. To regain good precision, moderately informative variance priors or additional 
mathematical assumptions may be necessary. 

Methods: In this paper we describe four novel approaches to modeling heterogeneity variance - two novel model 
structures, and two approaches for use of moderately informative variance priors. We examine the relative 
performance of all approaches in two illustrative MTC data sets. We particularly compare between-study 
heterogeneity estimates and model fits, treatment effect estimates and 95% credible intervals, and treatment rank 
probabilities. 

Results: In both data sets, use of moderately informative variance priors constructed from the pair wise 
meta-analysis data yielded the best model fit and narrower credible intervals. Imposing consistency equations on 
variance estimates, assuming variances to be exchangeable, or using empirically informed variance priors also 
yielded good model fits and narrow credible intervals. The homogeneous variance model yielded high precision at 
all times, but overall inadequate estimates of between-trial variances. Lastly, treatment rankings were similar among 
the novel approaches, but considerably different when compared with the homogenous variance approach. 

Conclusions: MTC models using a homogenous variance structure appear to perform sub-optimally when 
between-trial variances vary between comparisons. Using informative variance priors, assuming exchangeability or 
imposing consistency between heterogeneity variances can all ensure sufficiently reliable and realistic heterogeneity 
estimation, and thus more reliable MTC inferences. All four approaches should be viable candidates for replacing or 
supplementing the conventional homogeneous variance MTC model, which is currently the most widely used in 
practice. 
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Background 

Multiple treatment comparison (MTC) meta-analysis is an 
extension of conventional pair wise meta-analysis where 
only two interventions are being compared at the time. In 
contrast to pair wise meta-analysis, MTCs allow for simul- 
taneous inferences about the comparative effectiveness and 
safety of multiple (3 or more) interventions. The statistical 
models used to analyze meta-analytic data on multiple 
interventions are commonly employed in the Bayesian fra- 
meworks [1] and conventionally employ non-informative or 
weakly informative priors for all model parameters (e.g., 
treatment effects and heterogeneity variances). Such priors 
are preferred for two main reasons. First, readers are typic- 
ally already familiar with the purely data driven frequentist 
approach for pair wise meta-analysis, and use of non- 
informative or weakly informative priors allows the analysis 
to, at least theoretically, remain data driven. Second, there 
is an unfortunate but prevailing concern about use inform- 
ative priors because such are believed to drive results in the 
direction of the researchers' personal believe. While use of 
informative priors elicited for treatment effect parameters 
may be inappropriate, it is a misconception that informative 
priors are necessarily inappropriate for other parameters. 
This is especially true for parameters where the immediate 
effect of the informative priors on the treatment effects is 
not apparent. 

Variance parameter estimates play an important role in 
the overall inferences of an MTC since they impact the 
width of 95% credible intervals and treatment rank prob- 
abilities. A largely under-recognized issue in random-effects 
MTCs (as well as Bayesian pair wise random-effects meta- 
analysis) is that apparently weakly informative heterogen- 
eity variance priors may often be moderately informative 
[2-4], and thus, bias overall inferences to a considerably lar- 
ger degree than a well thought out informative variance prior 
would [4-6]. This is particularly relevant in random-effects 
MTCs where the results of an analysis can change dramatic- 
ally depending on several factors including number of stud- 
ies, the amount of heterogeneity between studies [4,7-9]. 

Another under-recognized issue in random-effects MTCs 
is the importance of the assumptions made about the simi- 
larity and correlation between the degrees of heterogeneity 
across treatment comparisons (i.e., assumed heterogeneity 
variance structures) [4,10,11]. Random-effects MTCs have 
commonly been carried out under the assumption that the 
between-trial variances representing each of the treatment 
comparisons are equal (this assumption is also known as 
the common variance or 'homogeneous variance assump- 
tion) [12-14], This approach 'borrows strength' for hetero- 
geneity estimation across treatment comparisons, and so, 
the risk that a weakly informative variance prior uninten- 
tionally becomes moderately informative is mitigated. How- 
ever, the homogenous variance assumption is typically 
unrealistic because the heterogeneity variances are likely 



different across treatment comparisons [5,6,15]. As a result, 
95% credible intervals may not maintain their nominal 
coverage, and treatment rank probabilities may be distorted 
[10,15]. Of course, when employing weakly informative vari- 
ance priors, relaxing the homogeneous variance assumption 
may be equally problematic due to a reduction in precision 
for estimating heterogeneity across treatment comparisons. 

There are a number of approaches for eliciting or con- 
structing informative variance priors in random-effects 
MTCs. Further, there are a number of possible heterogen- 
eity variance structures under which weakly informative 
variance priors can be employed. To date, no comparison 
of the available informative and weakly informative 
approaches is available of their relative performance. In 
this article we review and compare six random-effects 
MTC models - four under which weakly informative vari- 
ance priors are elicited, and two under which moderately 
informative variance priors are elicited. The four weakly 
informative models include the conventional homoge- 
neous variance model, the unrestricted heterogeneous 
variance model, the exchangeable variances model, and 
the consistency variances model. The two moderately in- 
formative models are structurally based on the unre- 
stricted heterogeneous variance model and the variance 
priors are either frequentistic distribution approximations 
from within the MTC data or distributions previously 
derived from a large external empirical data set. We place 
comparative emphasis on the homogeneous variance 
model since this approach is conventionally used in MTC 
practice. We discuss how inferences from the informative 
approaches as well as the other weakly informative 
approaches theoretically line up against inferences from the 
conventional homogeneous variance MTC model. We 
compare treatment effect estimates 95% credible intervals, 
heterogeneity variance estimates and posterior distribu- 
tions, and treatment rank probabilities from the discussed 
models in two illustrative examples. MTC treatment effect 
and variance estimates are also compared with those from 
pair wise meta-analyses. 

Methods 

In this section we first describe, distinguish and discuss 
what is meant by different degrees of information con- 
tained in the prior distributions in Bayesian MTCs. We 
then describe the general MTC model setup, as well as the 
setup for the commonly applied homogeneous variance 
MTC model. Lastly, we describe six approaches to model- 
ling between-trial variances that make use of different 
combinations of heterogeneity variance parameterizations 
and priors. 

Prior information terminology 

In the introduction we mentioned use of non-informative; 
weakly informative; and moderately informative' priors. 
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These terms are often used vaguely or interchangeably in 
the literature. Below, we define, distinguish and discuss 
what exactly is meant in this article when priors are 
'non-informative', weakly informative', or 'moderately 
informative'. 



Non-informative priors 

In this article, we define non-informative priors' as prior 
distributions, that carry virtually no information about the 
likely true value of a parameter. For example, for treatment 
effects measured as log odds ratios in a logistic regression 
model (which is the typical set up for MTCs of binary data), 
a normal distribution with mean zero and variance 10000 
carries virtually no information about the likely true log 
odds ratio, and thus, constitutes a non-informative prior 
distribution. For a between-trial variance parameter, an ex- 
ample of a non-informative prior could be a gamma distri- 
bution with shape and scale parameters of both 10" 10 . It 
should be noted that because Bayesian analysis is typically 
realized by Markov Chain Monte Carlo (MCMC) sampling, 
which relies on prior distributions and initial sampling 
values being sufficiently reasonable to allow for conver- 
gence of the posterior distribution, there is a limit to how 
non-informative a prior can feasibly be. For example, run- 
ning the MCMC sampling for a Bayesian MTC may not be 
feasible if a gamma distribution with shape and scale para- 
meters of 10" 10 is used for the between-trial variance 
parameter. 

Weakly informative priors 

In this article, we define a weakly informative' prior as a 
prior distribution that carries more information than a 
non-informative prior, but deliberately carries smaller de- 
gree of information than is actually available. The purpose 
of using weakly informative priors rather than non- 
informative priors is typically to achieve some stabilization 
in the MCMC sampling and/or estimation procedure. In 
the context of MTCs, a typical example of a weakly inform- 
ative prior for the between-trial standard deviation param- 
eter is the conventionally used uniform distribution 
between 0 and 2 when data is dichotomous and treatment 
effects are modelled as log odds ratios (ie, modelled in a lo- 
gistic regression framework). This prior carries more infor- 
mation than a typical non-informative variance prior (e.g., 
the above mentioned gamma distribution). It is well known 
that between-trial variances on the log odds ratio scale gen- 
erally do no exceed a value of 4, and so, this knowledge is 
used by truncating the between-trial standard deviation to 
2. It is also known that between-trial variances on the log 
odds ratio scale are typically smaller than 1 and closer to 0. 
However, this knowledge is only used partially for this prior 
since the probability of observing larger between-trial 



variance values only decreases slightly for larger values [3]. 
The danger with using weakly informative variance priors 
in MTCs is that the data is often relatively sparse, and so, a 
variance prior that is presumed weakly informative can eas- 
ily become moderately and sometimes highly informative. 
For example, the expected value for the heterogeneity vari- 
ance from the above unifom prior is approximately 1.33. 
However, in a setting where the heterogeneity variance is 
likely to be close to 0 (e.g., very similar trials designs and 
drug responses do not differ much across populations) and 
where only a few small trials are available to inform the 
variance estimation, this prior may easily upward bias the 
heterogeneity variance estimate, and thus, create artificially 
wide credible intervals. 



Moderately informative priors 

In this article, we define a moderately informative' prior as 
a prior distribution that carries a distinguishable and larger 
degree of information than a weakly informative prior. The 
purpose of using a moderately informative prior is to either 
fully or partially mix prior (external) knowledge about one 
or more parameters with the data. To this end, the data still 
plays an important role. One example of a moderately in- 
formative prior is use of observational data about the mag- 
nitude of one or more comparative treatment effects. For 
example, if observational studies have suggested that one 
novel treatment exhibits a 25% reduction of symptoms over 
another novel treatment, one can use this evidence to pro- 
duce a mean parameter value (treatment effect) in the prior 
distribution and subsequently elicit a variance that corre- 
sponds to the weight and confidence one is willing to put 
in this value. Another example of moderately informative 
priors in the MTC framework is the use of empirical evi- 
dence on the distribution of between-study variance esti- 
mates across published meta-analyses. This is also the last 
of the six heterogeneous variance approaches considered in 
this article, and will be illustrated below. 



General MTC model set up 

For this manuscript we describe MTC models of binary 
data. However, the modelling concepts are easily extended 
for other types of data such as count data and continuous 
data [16]. For simplicity, we also assume that all trials 
included in an MTC are 2-arm trials. Multi-arm trials ne- 
cessitate modelling of correlations between treatment com- 
parisons with a common comparator. We refer to previous 
papers for detailed description of this issue [10,13]. 

In the binary data setting, a commonly used effect meas- 
ure in MTCs is the odds ratio (OR). For each treatment 
comparison, odds ratios are typically estimates with a logis- 
tic regression model that simultaneously links the trial-arm 
odds and the treatment comparison odds ratios. Letting k 
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denote the number of trials and T the number of treat- 
ments in a network, and letting t=l,. . .T indicate the treat- 
ment in focus and j=l,...k the trial in focus, then the 
following main distributional and deterministic relation- 
ships make up the core of the MTC model in the Bayesian 
framework 

r r bin(n jtPjt ) 
logit(^) = p jt + 8 jtb l [t>b] 
8 jtb ~N{d tb ;0 tb 2 ) 
dtb = d t i — d b i 

Where pj t is the probability of an event in trial / under 
treatment t> and Vj t and rij t are the number of events and 
the number of patients in the corresponding treatment 
arm; pj b is the log odds of having an event in the control 
arm (i.e., with 'baseline treatment' b) in trial 8j tb is the 
log odds ratio of treatment t relative to treatment b in 
trial d tb is the 'true' overall treatment effect of t relative 
to b, and o tb is the corresponding between-trial vari- 
ance. The last equation represents the consistency as- 
sumption, which is necessary for all MTC models, and 
dictates that any expected relative treatment effect of a 
direct (head-to-head) evidence source is equal to the 
corresponding expected relative treatment effect of an 
indirect evidence source. In other words, the consistency 
assumptions dictates that the results from direct and in- 
direct sources of evidence should not differ beyond the 
play of chance. 

In the above, the control arm (baseline) log odds para- 
meters fijb are treated as nuisance parameters, and assigned 
non-informative normal distribution priors with mean 0 
and very large variances, typically of 1000 or 10000. For 
b=l, the overall log odds ratios d tb (i.e., the treatment effect 
of t) are also assigned non-informative normal distribution 
priors with mean 0 (representing no effect) and large var- 
iances, typically of 1000 or 10000. 

MTC models with weakly informative variance priors 
The homogeneous variance model 

Under the homogenous variance MTC model the assump- 
tion is made that all between-trial variances are equal. That 
is, strictly speaking we assume a tb =cf for all treatment 
comparisons t versus b, or specifically, that the between- 
trial variance for all treatment comparisons is equal to o 2 . 

Typically a weakly informative prior is assigned to a 
(the between-trial standard deviation) under the homo- 
geneous variance model. Although a number of weakly 
informative variance priors have been used throughout 
the MTC literature (e.g., gamma distribution or half- 
normal), the most commonly used variance priors are 
weakly informative uniform distributions between 0 and 
2 or between 0 and 10 [13,16]. 



The unrestricted heterogeneous variances model 

Under the heterogeneous variance MTC models, all 
between-trial variances are allowed to take on different 
values. The unrestricted heterogeneous variances model 
places no structural restrictions on the heterogeneity var- 
iances. Under this model, weakly informative priors can be 
assigned to each of the between-trial variance parameters 
o tb . Conventionally, one would make use of the uniform 
distribution from 0 to 2 or from 0 to 10 as prior distribu- 
tions for the between-trial standard deviations. The het- 
erogeneous variance model with such priors is typically 
referred to as the unrestricted heterogeneous variance 
model. 

Theoretically, this model is advantageous due to its high 
flexibility in modelling heterogeneity variances. In practice, 
however, this model is often sub-optimal because many 
comparisons are typically only informed by a few trials, and 
thus, the estimation of between-trial variances (i.e., their 
posterior distributions) is very imprecise. The below four 
Bayesian modelling approaches are modifications of the 
unrestricted heterogeneous variance model that apply dif- 
ferent parameter value constraints or moderately inform- 
ative prior distributions to optimize the estimation of the 
between-trial variance parameters. 

The exchangeable variances model 

One approach to gaining precision for the between-trial 
variance estimation is to meet in the middle' between the 
homogeneous and heterogeneous variances models by as- 
suming that the between-trial variances are exchangeable. 
That is, one can assume that the between-trial variances 
are random samples from a common between-trial distri- 
bution, thus allowing them to borrow strength from each 
other [2]. In particular, one would assume some common 
precision parameter, <x, and then sample between trial var- 
iances from any treatment comparison t vs b from a trun- 
cated t-distribution with df the degrees of freedom (the 
number of trials for the comparison of treatment t vs b 
minus 1) 

<Ttb~t(Q, d£),o--G tb >0 

Here we assign a weakly informative prior distribution to 
the common' between-trial variance corresponding to the 
common precision,' (Ho) ~ U(0,2). The prior distributions 
for the individual between-trial variances, o tb , can be 
thought of as weakly informative due to the reliance on the 
common variance' parameter and the degrees of freedom. 
We refer to this approach as the exchangeable variances 
MTC model 

Theoretically, the exchangeable variances MTC model 
gains the best of two worlds. It gains precision by borrow- 
ing strength from the common variance assumption, but 
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it retains flexibility in allowing for differing between-trial 
variances. In practice, however, this model may not per- 
form optimally when the between-trial variances differ 
considerably across comparisons. This is because the as- 
sumption of a common variance ties all individual 
between-trial variances probalistically to some central ten- 
dency, in which case heterogeneity parameters that are 
truly not close to the central tendency will be inaccurately 
estimated. Arguably, the exchangeable variance approach 
may work best in situations where 1) the interventions 
being investigated in the MTC are all similar (e.g., of the 
same drug class or solely pharmacotherapies); and 2) the 
study designs and patient eligibility criteria are fairly 
comparable. 

The heterogeneous variances model using second order 
consistency inequalities 

Another approach to gaining precision but retaining flexi- 
bility in modelling of heterogeneous variances is to re- 
parameterize the variance structure in order to ensure that 
the property of consistency also holds for the between-trial 
variance (and between-comparison correlation) parameters 
[10]. The consistency relationship for variances is as fol- 
lows. For any three treatments b, x, and y, we assume 
consistency. That is, for the three corresponding (mean) 
comparative treatment effects d yx) d yb) and d xb) we assume 
that 

dyx — dy\y dxb 

This equation is also sometimes referred to as the first 
order consistency equation. Taking the variances of each 
side of the above equation we have 

2 2 2 \) 2 2 

0~ yx = Oyb + °~xb — 2p yx O y b0~xb ^ G yb + &xb 

Where a yx , o yb and a xb , are the variances of d yxt d yb , and 
d xb , respectively, and p yx b is the correlation between d yb) 
and d xb . The above equation implies a second order 
consistency triangle inequality 

12 21 2 12 21 

\0~yb —0~xb\ ^ 0-yx < \O yb +a x b \ 

Where \x\ denotes the absolute value of any variable, x. 
This inequality can be incorporated in the model to restrict 
the variance and correlation parameters to plausible pos- 
sible values and allow for better adherence to consistency. 
However, incorporating the consistency triangle inequality 
in the conventional heterogeneous variance MTC model 
can create serious difficulties in assigning appropriate 
priors. To solve this issue, Lu and Ades proposed a re- 
parameterization of the heterogeneous variance model in 
which each between-trial variance parameter would be 
represented by the sum of variances of the two involved 
treatment arms minus the corresponding covariance [10]. 



The resulting covariance matrix is represented as the prod- 
uct of variance vectors and a correlation matrix, where the 
correlation matrix is constructed via a Cholesky decompos- 
ition using spherical coordinates to allow for weakly in- 
formative priors. We refer to the paper by Lu and Ades for 
the mathematical details [10]. For the remainder of this 
paper we refer to the above approach as the consistency 
variances MTC model 

Theoretically, the consistency variances model is optimal 
in that it largely retains the flexibility of the unrestricted 
variances model, and additionally restricts variances in 
alignment with and borrows strength from the seminal as- 
sumption of consistency. In practice, the consistency tri- 
angular inequality may not hold within the available data 
since between-trial variance estimates (and posterior distri- 
butions) may fluctuate and differ due to the play of chance 
[17], time-dependent biases [18], and binary event rates 
[19]. Incorporating the consistency triangular inequality 
imposes an adjustment to the variances if the inequality is 
not met within the data, but there is no guarantee that this 
adjustment is in the right direction. 

MTC models with moderately informative variance priors 

Considering the limitation of the above models, one could 
argue that random-effects MTCs incorporating sensible 
moderately informative variance priors constitute a viable 
alternative. Below we propose two sensible approaches for 
obtaining and eliciting informative variance priors in 
random-effects MTCs. 

Using frequentist within-data approximate distribution as 
priors 

Informative variance priors should aid in ensuring that the 
estimation of between-trial variances is directed with ap- 
propriate probability mass to plausible intervals of possible 
values. It therefore seems reasonable to require that vari- 
ance estimates and their posterior probability distributions 
should be directed towards the values one would have 
obtained in separate pair wise meta-analysis, and vice versa 
[11]. We therefore put forward, that the probability distri- 
butions for the between-trial to variance estimated from 
the available data in a frequestist framework could readily 
be used as informative variance priors in MTCs. While a 
number of methods are available for estimating variance 
distributions, we particularly consider the approximate 
gamma distribution proposed by Biggerstaff and Tweedie 
[20], albeit in a modified version to fit MTC modeling. This 
frequentist approximate distribution is a location-shifted, 
scaled gamma distribution for the DerSimonian-Laird (DL) 
estimator, o~ D l, based on the relationship between this esti- 
mator and Cochrans Q (test for heterogeneity), o D t = (Q- 
(k-l))/(Sj - (S^Sj)), where k is the number of trials, S 2 is 
the sum of trial weights (ie, inverse variances) and S 2 is the 
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sum of squared trial weights [21]. With respect to the two 
treatments being compared, x and y, the approximate 
gamma distribution of Q and its parameters are given 

Qyx~E (r yx^yx) 

r yx = E(Q yx ) 2 /Var(Q yx ) 
k yx = E(Q yx )/Var(Q yx ) 

Where E(Q yx ) and Var(Q yx ) is the expected value and 
variance of Q^. We refer to the paper by Biggerstaff and 
Tweedie for the approximate deterministic expressions 
of E(Q yx )zn& Var(Q yx ) [20]. 

While the approximate distribution for a D l for any com- 
parison is a candidate as an informative variance prior, it 
does have some undesirable limitations in the context of 
Bayesian analysis. First, o D l can yield negative estimates 
and will in this case be truncated to 0 [21]. If used as a 
variance prior in the Bayesian framework, this property 
may create a bi-modality on the posterior distribution. 
Such a bi-modality may increase the time to convergence 
of the Markov Chain Monte Carlo (MCMC) sampling and 
result in poor model fits (ie, large deviance information 
criterion, DIC). Another issue is the well-known tendency 
of a D l to underestimate the between-trial variance 
[7,8,22]. To avoid these issues, we propose to use a con- 
sistently positive estimator proposed by Hartung and 
Makambi (HM) [23]. In contrast with the DL estimator, 
which is derived as a 1 st order method of moments esti- 
mator, the HM estimator is a 2 nd order method of 
moments based estimator and has the following expres- 
sion 

ohm 2 = Q((2(* - 1) + Q)(S X - (S 2 /Si)) 

The HM estimator is consistently positive and has been 
shown to yield accurate and precise estimates of the 
between-trial variance [7,9,23]. HM is a function of Q, and 
thus, by incorporating the prior distribution of Q in the 
WinBUGS code and subsequently deriving o H m via its ori- 
ginal expression, the shortcomings of the DL approach are 
circumvented. 

The above proposed approach for obtaining and eliciting 
informed variance priors is either optimal or sub-optimal 
depending on the assumptions one is willing to make. By 
informing variance estimation with prior distributions cor- 
responding to the expected likelihood in a frequentist ana- 
lysis, one imposes a '2-stage' estimation process that lets 
the Bayesian MCMC sampling concentrate' on the esti- 
mation of treatment effects. An analogous process was re- 
cently proposed in the purely frequentist framework [11]. 
The informed variance prior approach, however, is sub- 
optimal if one is not willing to believe the frequentist 
variance likelihoods and prefers to incorporate additional 
uncertainty around variance estimation. Further, approxi- 



mating the heterogeneity variance distributions as sug- 
gested above, may be work intensive. 

Heterogeneous variances using empirically derived 
informative priors 

A simpler and more general approach to incorporating 
informed variance priors is to borrow strength from exter- 
nal empirical evidence. Turner et al. reviewed 14886 
Cochrane Database meta-analyses including a total 77237 
trials and approximated the empirical distribution of the 
between-trial variance categorized by type of outcome 
(mortality, semi-objective and subjective), type of interven- 
tion, and field of medicine [6]. The mean and variance par- 
ameter values for log-normal distributions were estimated 
by category [6]. These empirically derived log-normal distri- 
butions can readily be used as moderately informative vari- 
ance priors under the unrestricted heterogeneous variance 
model. For example, Turner et al. empirically approximated 
the heterogeneity variance distribution for meta-analyses 
comparing pharmacological interventions on subjective 
outcomes (e.g., dichotomous biomarker outcome) to a log- 
normal distribution with mean -2.34 and variance 1.62 [2]. 
In an MTC comparing only pharmacological interventions 
on a subjective outcome (as is the case in illustrative ex- 
ample 1), one can then elicit this log-normal distribution 
for all heterogeneity variance parameters instead of the 
conventional weakly informative uniform distribution. 

This informative variance approach is relatively straight- 
forward to apply. The already empirically approximated 
priors have general applicability due to the sample size of 
the empirical study from which they originated. However, 
to the extent other factors than the ones explored by 
Turner et al. determine the likely degree and distribution of 
heterogeneity variance, the approach may not produce opti- 
mal variance estimation. 

Results 

We applied the above considered models and priors to two 
MTC data sets of differing size and complexity to illustrate 
the performance. The treatment networks for our two 
examples are presented in Figure 1. We compared the 
inferences from the five described heterogeneous variance 
MTC models with the homogeneous variance MTC model 
and with reference to the heterogeneity estimates obtained 
from pair wise meta-analysis. In particular, we compared 1) 
the model fit (using the deviance information criterion 
(DIC)) as well as the estimates and posterior distributions 
of the between-study heterogeneity variances; 2) the magni- 
tude, direction and significance of each treatment compari- 
son; and 3) the ranking of the treatments in terms of 
probabilities of being the best treatment. 

The DIC is a measure of model fit computed from the 
likelihood function with a penalty for complexity [24]. The 
complexity is measured as the effective number of 
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parameters,' which is abbreviated pD' [24]. The DIC is simi- 
lar to the AIC and BIC, and a lower value means a better 
fit [24]. The probability of 'being the best treatment' is 
derived as the probability of being the largest odds ratio 
among MCMC simulations from the posterior distribution. 

We compared the heterogeneity variances from all 
MTC models with the DerSimonian-Laird and Hartung- 
Makambi estimates from pair wise meta-analyses, as well 
as with the Bayesian pair wise meta-analysis estimates. 
Considering the pair wise heterogeneity variance estimates 
as the bench mark, we then assessed the extent to which 
observed differences in inferences between MTC models 
could be explained by poor estimation of between-study 
heterogeneity variances and their posterior distributions. 

All Bayesian MTC models were carried out in WinBUGS 
v. 1.4.3 [25]. Convergence of Markov Chain Monte Carlo 
simulation was assessed using the Brooks-Gelman-Rubin 
criteria using 3 chains, and based on the findings of the 
convergence analysis, a burn-in of 20000 iterations was 
used for all MTC analysis. Similarly, MTC model inferences 
were based on 20000 iterations following the burn-in 
period. Frequentist meta-analyses were carried out in 
Rv2.U [26]. 



Illustrative example 1 

In our first example, we use data from two Cochrane Data- 
base systematic reviews on interventions for treating hepa- 
titis C [27,28]. The MTC data set is a simple fully 
connected treatment network of the three interventions: 
Peglnterferon alpha-2a plus Ribavirin (PEG-2a+RBV), 
Peglnterferon alpha-2b plus Ribavirin PEG-2b+RBV), and 
standard Interferon + Ribavirin (INF+RBV) (see Figure la). 
The population is limited to treatment-naive patients and 
excludes patients with co-infections (e.g., HIV). We use the 
meta-analysis data for the conventionally used surrogate ef- 
ficacy measure sustained virologic response (SVR). 

In this data set, each of the three treatment comparisons 
is informed by a comparable amount of evidence. In par- 
ticular, the comparison of PEG-2a+RBV and INF+RBV 



includes 4 trials and 1197 patients, the comparison of PEG- 
2b+RBVand INF+RBV includes 12 trials and 2750 patients, 
and the comparison of PEG-2a+RBV and PEG-2b+RBV 
includes 6 trials and 2994 patients. The trials in the three 
comparisons (pairwise meta-analyses) each incurred differ- 
ent degrees of heterogeneity (e.g., DerSimonian-Laird be- 
tween-trial variance estimates of 0.64, 0.00, and 0.04). This 
suggests a need for modelling the between-trial variances as 
heterogeneous in the MTC model, which makes this data 
set a good candidate for how well the heterogeneous vari- 
ance MTC models perform in this context and how they 
measure up against the conventional homogeneous vari- 
ance model. For the empirically informed variances' model 
we used a log-normal distribution with mean -2.34 and 
variance 1.62 [2] because all interventions being compared 
are pharmacological and the outcome, SVR, is a dichotom- 
ous biological marker, which fits under subjective outcome' 
definition by Turner et al. [6]. 

As expected, the homogeneous variance MTC models 
yielded a worse model fit than the heterogeneous variance 
MTC models according to the DIC (Table 1). The informed 
variance model based on frequentist approximate distribu- 
tions yielded the best model fit according to the DIC. The 
remaining four heterogeneous variance models yielded 
comparable DICs. Comparison of the common' between- 
trial variance estimate with the frequentist estimates as well 
as the estimates from the five heterogeneous variances 
MTC models strongly suggests that the 'homogeneous vari- 
ance' assumption is both strongly violated and will result in 
an unrealistic between-trial variance estimates for most 
(if not all) comparisons (Table 1). Among the five heteroge- 
neous variances MTC model, the informed variances model 
based on frequentist approximate distributions produced 
variance estimates closest to the frequentist ones and had 
the posterior variance distributions with the highest preci- 
sion (Figure 2). The empirically informed variances model 
had the second highest posterior distribution precision, the 
consistency variances model third, the exchangeable var- 
iances model fourth, and lastly the unrestricted variances 
model fifth (Figure 2). 



PEG-2A+RBV PEG-2B+RBV 
4 



Trt2 



Trt3 




Trtl 6 40 Trt4 

39 \ / 8 




INF+RBV 



Control 



Figure 1 Presents the treatment networks with the number of trials informing each treatment comparison in our two illustrative 
examples. The treatment network on the left is the network for our first illustrative example. The treatment network on the right side is the 
network for our second illustrative example. The circles represent the treatments in the network, the lines represent the comparisons where 
head-to-head (direct) evidence is available, and the numbers in the lines present the number of randomized clinical trials available per 
comparison. Abbreviations: PEG-2A (Peginterferon-2a); PEG-2B (Peginterferon-2b); INF (Interferon), RBV (Ribavirin); Trt (Treatment). 
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For the comparison between peginterferon-2a and inter- 
feron and the comparison between the two peginterferons, 
the homogeneous variance model has narrower 95% cred- 
ible intervals that all other heterogeneous variance models, 
except for the informed variance model based on frequen- 
tist approximate distributions (see Table 2). For the com- 
parison between peginterferon-2a and interferon, the 
homogeneous variance model yielded a comparably wider 
95% credible interval (see Table 2). The unrestricted var- 
iances model had the widest credible intervals among the 
heterogeneous variances models, and the informed var- 
iances model based on frequentist approximate distribu- 
tions had the narrowest credible intervals. Because this 
network only included three treatments we did not calcu- 
late treatment rank probabilities. 

Illustrative example 2 

Our second example data set is a larger, more diverse treat- 
ment network including four pharmacological interventions 
(Trtl, Trt2, Trt3, and Trt4) and a control for cessation of a 
harmful behaviour (See Figure lb) [15]. In this example the 
outcome of interest is taken at 6 months follow-up. The 
included studies all enrolled participants at initiation of 
therapy. Each of the four interventions had been compared 
to control, and the first two had been compared to each 
other. The amount of evidence differed across comparisons. 
In particular, Trtl versus placebo was informed by 39 trials 
and 16674 patients, Trt2 versus placebo was informed by 6 
trials and 3222 patients, Trt3 versus placebo was informed 



by 40 trials and 10682 patients, and the Trt4 versus placebo 
was informed by 8 trials and 3678 patients, and lastly, Trt 2 
vs Trt 1 was informed by 4 trials and 2330 patients. These 
five head-to-head comparisons (pairwise meta-analyses) in- 
curred only moderately different degrees of heterogeneity, 
except for Trt3 versus placebo where little to no heterogen- 
eity was detected (see Table 3). This suggests the homoge- 
neous variance model may not perform too poorly. 
However, the situation still raises uncertainty about 
which model is most suitable and therefore warrants 
modelling with a proposed heterogeneous variance 
models for the purpose of identifying the best fit (and 
thus most valid inferences). For the empirically informed 
variances' model we used a log-normal distribution with 
mean -3.02 and variance 1.85 [2] because all placebo 
comparisons and a log-normal distribution with mean 
-3.23 and variance 1.88 [2] for comparison of active inter- 
ventions, since all interventions being compared are 
pharmacological and the outcome, cessation to a harmful 
behavious, fits under the semi-objective outcome' defin- 
ition by Turner et al [6]. 

According to the DIC, the informed variances model based 
on the frequentist approximate variance distributions yielded 
the best model fit (Table 3). The homogeneous variance 
model and the remaining four heterogeneous variances 
models yielded similar model fits according to the DIC 
(Table 3). Comparison of the common' between-trial vari- 
ance estimate with the frequentist estimates as well as the 
estimates from the five heterogeneous variances MTC mod- 
els suggests that the 'homogeneous variance' assumption 



Table 1 Between-trial variance estimates and model fit statistics from the considered models and priors in the first 
illustrative example on hepatitis C treatments for achieving sustained virological response (SVR) 







Between-trial variance estimates 




Model fit statistics 


Models 


PEG-2A+RBV vs INF+RBV 


PEG-2B+RBV vs INF+RBV PEG-2B+RBV vs PEG-2A+RBV 


pD 


DIC 


Random-effects pair wise meta-analysis 


Frequentist (DerSimonian-Laird) 


0.642 


0.000 


0.036 






Frequentist (Hartung-Makimbi) 


0.580 


0.017 


0.021 






Bayesian (weakly informative) 


0.700 


0.018 


0.077 






Bayesian (frequentist informed) 


0.422 


0.001 


0.038 






Bayesian (empirically informed) 


0.091 


0.024 


0.052 






Random-effects MTC models 


Weakly informed variance models 


Homogeneous variance model 


0.097 


0.097 


0.097 


33.3 


283.2 


Unrestricted variances 


1.046 


0.017 


0.103 


32.8 


277.7 


Exchangeable variances 


0.510 


0.016 


0.083 


32.6 


278.6 


Consistency variances structure 


0.225 


0.024 


0.164 


32.9 


280.2 


Moderately informed variance models 


Frequentist informed priors 


0.677 


0.011 


0.047 


30.9 


275.6 


Empirically informed priors 


0.368 


0.026 


0.076 


32.4 


278.4 



Abbreviations: PEG-2A (Peginterferon-2a); PEG-2B (Peginterferon-2b); INF (Interferon), RBV (Ribavirin). 
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Figure 2 Presents the posterior distributions of the between-trial variance parameters in the first illustrative example under the six 
employed MTC models: the homogeneous variance model (row 1); the unrestricted variances model (row 2); the exchangeable 
variances model (row 3); the consistency variances model (row 4); the frequentistically informed variances model (row 5); and the 
empirically informed variances model (row 6). The two presented comparisons are: peginterferon-2a (PEG-2A) vs Interferon (INF) (column 1), 
and Peginterferon-2a (PEG-2A) vs Peginterferon-2b (PEG-2B) (column 2). The comparison of PEG-2B vs INF was selective excluded due to the 
posterior variance distributions being more similar across the five heterogeneous variance approaches. 



is mildly to moderately violated. Among the five heteroge- 
neous variances MTC model, the consistency variance model 
and the informed variances model using frequentist approxi- 
mate distributions produced estimates closest to the frequen- 
tist ones. The exchangeable variance model and the 
empirically informed variances model also produced seemingly 



reliable variance estimates. Again, the informed variance 
model using frequentist approximate distributions had the 
highest posterior distribution precision (Figure 3). The em- 
pirically informed variances model had the second highest 
posterior distribution precision, the consistency variances 
model third, the exchangeable variances model fourth, 



Thorlund et al. BMC Medical Research Methodology 2013, 13:2 
http://www.biomedcentral.eom/1471-2288/13/2 



Page 10 of 14 



Table 2 Odds ratios and 95% confidence/credible intervals for the three comparisons from the considered models and 



priors in the first illustrative example on hepatitis C treatments for achieving sustained virological response (SVR) 



Model 


PEG-2A+RBV vs INF+RBV 


PEG-2B+RBV vs INF+RBV 


PEG-2B+RBV vs PEG-2A+RBV 


Random-effects pair wise meta-analysis 


Frequentist (DerSimonian-Laird) 


3.63(1.51-8.73) 


1.30(1.11-1.52) 


1.38(1.07-1.79) 


Frequentist (Hartung-Makimbi) 


3.60(1.56-8.34) 


1.35(1.11-1.64) 


1 .38(0.36-3.24) 


Bayesian (weakly informative) 


NA* 


1.36(1.10-1.73) 


1 .38(0.94-2.22) 


Bayesian (frequentis informed) 


NA* 


1.30(1.11-1.55) 


1 .40(0.86-2.47) 


Bayesian (empirically informed) 


NA* 


1.36(1.10-1.74) 


1.38(1.02-1.99) 


Random-effects MTC models 


Weakly informed variance models 


Homogeneous variance model 


2.42(1.75-3.60) 


1.53(1.19-2.03) 


1.58(1.18-2.26) 


Unrestricted variances 


2.11(1.40-3.57) 


1.38(1.13-1.79) 


1.50(1.06-2.53) 


Exchangeable variances* 


2.17(1.48-3.43) 


1.40(1.15-1.79) 


1.53(1.11-2.36) 


Consistency variances structure 


2.39(1.63-3.80) 


1.42(1.16-1.86) 


1.67(1.17-2.68) 


Moderately informed variance models 


Frequentist informed priors 


2.04(1.45-2.93) 


1.38(1.15-1.69) 


1.46(1.12-2.05) 


Empirically informed priors 


2.23(1.54-3.40) 


1.44(1.16-1.83) 


1.54(1.13-2.29) 



* The MCMC simulation did not converge for the log odds ratio parameter (within the first 1.000.000 runs), and thus did not produce meaningful results. 
Abbreviations: PEG-2A (Peginterferon-2a); PEG-2B (Peginterferon-2b); INF (Interferon), RBV (Ribavirin). 



and lastly the unrestricted variances model produced the 
most imprecise posterior distributions. Figure 3 Presents 
the posterior distributions of the between-trial variance 
parameters in the second illustrative example under the 
six employed MTC models: the homogeneous variance 
model (row 1); the unrestricted variances model (row 2); 



the exchangeable variances model (row 3); the consistency 
variances model (row 4); the frequentistically informed 
variances model (row 5); and the empirically informed var- 
iances model (row 6). The three presented comparisons 
are: Treatment 2 (Trt2) versus control (column 1); treat- 
ment 4 (Trt2) versus Control; and Trt4 versus Trtl. The 



Table 3 Between-trial variance estimates (posterior distribution median) for the comparisons that were also informed 
by head-to-head evidence in the treatment network in the second illustrative example 







Between-trial variance estimates 




Model fit statistics 


Models 


Trtl vs Placebo 


Trt2 vs Placebo 


Trt3 vs Placebo 


Trt4 vs Placebo 


Trt2 vs Trtl 


pD 


DIC 


Random-effects pair wise meta-analysis 


Frequentist (DerSimonian-Laird) 


0.086 


0.110 


0.016 


0.075 


0.106 






Frequentist (Hartung-Makimbi) 


0.083 


0.103 


0.040 


0.072 


0.112 






Bayesian (weakly informative) 


0.100 


0.371 


0.023 


0.103 


0.334 






Bayesian (frequentist informed) 


0.087 


0.121 


0.036 


0.067 


0.093 






Bayesian (empirically informed) 


0.088 


0.110 


0.021 


0.054 


0.059 






Random-effects MTC models 


Weakly informed variance models 


Homogeneous variance model 


0.078 


0.078 


0.078 


0.078 


0.078 


138.8 


1229.1 


Unrestricted variances 


0.100 


0.469 


0.023 


0.104 


0.214 


138.3 


1229.9 


Exchangeable variances* 


0.092 


0.226 


0.009 


0.066 


0.047 


133.7 


1232.2 


Consistency variances structure 


0.091 


0.172 


0.033 


0.075 


0.133 


136.6 


1230.0 


Moderately informed variance models 


Frequentist informed priors 


0.087 


0.172 


0.036 


0.069 


0.064 


135.6 


1226.4 


Empirically informed priors 


0.087 


0.199 


0.020 


0.054 


0.042 


133.8 


1229.5 



* The'average' variance was 0.173. 

Abbreviations: DIC (Deviance information criterion); pD (effective number of model parameters); Trt (Treatment). 



Thorlund et al. BMC Medical Research Methodology 2013, 13:2 
http://www.biomedcentral.eom/1471-2288/13/2 



Page 11 of 14 



remaining comparisons were selective excluded due to the 
posterior variance distributions being more similar across 
the five heterogeneous variance approaches. 

The treatment effect estimate and 95% credible inter- 
val for Trt2 were considerably affected by the variance 
assumption, and thus, so were indirect comparisons be- 
tween Trt2 versus other interventions (Table 4). The 
treatment effect estimate of Trt2 versus placebo was 



smallest with the homogeneous variance model, and the 
95% credible intervals were narrow compared with those 
of the heterogeneous variances models. These differences 
considerably impacted treatment rank probabilities. While 
Trtl and Trt3 consistently received very low rank prob- 
abilities (e.g., 0.5% chance of being the best), the probabil- 
ity of Trt2 versus Trt4 being the best treatment varied 
from 71.3% versus 28.2% with the homogeneous variance 



Trt2 vs Control Trt4vs Control Trt4vsTrtl 




0.0 0.5 1.0 0.0 0.5 1.0 0.0 0.5 1.0 



Figure 3 Presents the posterior distributions of the between-trial variance parameters in the second illustrative example under the six 

employed MTC models: the homogeneous variance model (row 1); the unrestricted variances model (row 2); the exchangeable 

variances model (row 3); the consistency variances model (row 4); the frequentistically informed variances model (row 5); and the 

empirically informed variances model (row 6). The three presented comparisons are: Treatment 2 (Trt2) versus control (column 1); treatment 4 

(Trt2) versus Control; and Trt4 versus Trtl. The remaining comparisons were selective excluded due to the posterior variance distributions being 

more similar across the five heterogeneous variance approaches. 
\ ) 
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model to informed variance model to 43.2% versus 56.8% 
with the unrestricted variance model (see Table 5). 

In this example, a number of reasons suggest the 
informed variances model based on frequentist approxi- 
mate variance distributions is the more optimal choice. 
First, this model clearly yields the best model fit according 
to the DIC. Second, it produces the variance estimates clos- 
est to those of the frequentist pair wise meta-analyses. 
Lastly, the full MTC from which this example is borrowed, 
the efficacy of the considered interventions was also investi- 
gated for 1 month, 3 months, and 12 months follow-up. 
For these outcomes, many of the comparisons were non- 
significant (i.e., the 95% credible intervals included 1.00) 
with the homogeneous variance model despite clear statis- 
tical significance in the pair wise meta-analyses. When we 
used variance priors informed by frequentist approximate 
variance distributions, this statistical significance was 
recovered. 



Discussion 

The variance structure in an MTC is challenging to esti- 
mate because it rests on the amount of evidence and the 
linkage between comparisons. A number of approaches are 
available, but their performance is tied with the appropri- 
ateness of the assumed linkage between comparisons, and 
in the Bayesian framework, the elicited variance priors. 
Conventional MTC models have made use of the unrealis- 
tic assumption that the between trial variances for the 
included comparisons are all equal [4-6,10,15]. Emerging 
evidence (including our examples), however, suggest this 



Table 5 Treatment rankings, the probability of being the 
best treatment, under the considered Bayesian 



Model and prior 


Trtl 


Trt2 


Trt3 


Trt4 


Weakly informed variances models 


Homogeneous variance 


0.00% 


28.2% 


0.00% 


71.8% 


Unrestricted variances 


0.01% 


56.8% 


0.00% 


43.2% 


Exchangeable variances 


0.01% 


49.3% 


0.02% 


50.4% 


Consistency variances structure 


0.04% 


45.0% 


0.01% 


55.9% 


Moderately informed variances models 


Frequentist informed priors 


0.00% 


42.5% 


0.00% 


57.5% 


Empirically informed priors 


0.00% 


46.5% 


0.00% 


53.5% 



MTC models for the second illustrative example. 



approach is sub-optimal [10,15]. Instead, there is a need to 
consider 'heterogeneous variance structures'. Because the 
amount of evidence to reliably estimate heterogeneity vari- 
ance parameters is typically sparse, some precision can be 
gained either by incorporating informative variance priors 
or by using alternative restrictive heterogeneity variance 
structures in connection with weakly informed variance 
priors. In this paper we have considered two types of in- 
formative variance priors: frequentist and empirically 
informed; and we considered two restrictive variance struc- 
tures with weakly informative priors: the exchangeable var- 
iances approach, and the consistency variances approach. 

Our examples suggest that these four approaches all 
allow for reliable estimation of differing between-study het- 
erogeneity variances across comparisons, whereas the unre- 
stricted approach often does not. To this end, these four 



Table 4 Odds ratios and 95% confidence/credible intervals for the four placebo comparisons and two select active 
intervention comparisons in the second illustrative example 



Model Trtl vs Placebo 


Trt2 vs Placebo 


Trt3 vs Placebo 


Trt4 vs Placebo 


Trt2 vs Trtl 


Trt4 vs Trt2 


Random-effects pair wise meta-analysis 


DerSimonian-Laird 


1.94(1.67-2.24) 


2.11(1.42-3.13) 


1.78(1.60-1.97) 


2.86(2.21-3.71) 


1.90 (1.17-3.09) 




Hartung-Makimbi 


1.94(1.68-2.23) 


2.09(1.42-3.09) 


1.77(1.57-2.00) 


2.86(2.21-3.70) 


1.91 (1.16-3.13) 




Bayesian (non-informative) 


1.98(1.70-2.31) 


2.38(1.30-5.82) 


1.80(1.61-2.02) 


2.89(2.08-4.06) 


1 .97 (0.66-8.88) 




Bayesian (frequentist informed) 


1.97(1.71-2.28) 


2.16(1.48-3.68) 


1.80(1.60-2.02) 


2.89(2.23-3.77) 


1.79 (1.17-3.55) 




Bayesian (empirically informed) 


1.97(1.71-2.28) 


2.13(1.47-3.98) 


1.80(1.61-2.03) 


2.89(2.20-3.80) 


1.84 (1.17-3.53) 




Random-effects MTC models 


Weakly informed variance models 


Homogeneous variance model 


1.91(1.67-2.19) 


2.59(1.97-3.50) 


1.80(1.57-2.08) 


2.90(2.23-3.79) 


1.36 (1.02-1.84) 


1.11(0.74-1.63) 


Unrestricted variances 


1.95(1.69-2.28) 


3.05(1.84-5.50) 


1.81(1.62-2.01) 


2.90(2.07-4.07) 


1 .56 (0.94-2.75) 


0.94(0.49-1.74) 


Exchangeable variances* 


1.93(1.67-2.26) 


2.89(1.98-4.43) 


1.78(1.56-1.99) 


2.89(2.17-3.86) 


1.50 (1.03-2.26) 


1.01(0.59-1.58) 


Consistency variances structure 


1.93(1.66-2.23) 


2.80(1.92-4.69) 


1.80(1.60-2.02) 


2.90(2.18-3.86) 


1 .45 (0.99-2.40) 


1.03(0.58-1.66) 


Moderately informed variance models 


Frequentist informed priors 


1.93(1.68-2.22) 


2.80(1.98-4.10) 


1.80(1.60-2.03) 


2.90(2.22-3.78) 


1.46 (1.02-2.11) 


1.05(0.66-1.61) 



Empirically informed priors 1 .93(1 .67-2.23) 2.84(2.00-4.26) 1 .80(1 .61-2.02) 2.91 (2.22-3.81 ) 1 .48 (1 .00-2.20) 1 .02(0.63-1 .59) 



Abbreviations: DIC (Deviance information criterion); pD (effective number of model parameters); Trt (Treatment). 



Thorlund et al. BMC Medical Research Methodology 2013, 13:2 
http://www.biomedcentral.eom/1471-2288/13/2 



Page 13 of 14 



approaches seem superior to the homogeneous variance 
structure model as well as the unrestricted heterogeneous 
variances approach. The frequentist informed approach 
yielded the best model fits in both example, and although 
further research is needed at this point, one could argue for 
this approach as a primary supplement to the conventional 
homogeneous model 

Our study offers several strengths, but also has some lim- 
itations. Our chosen illustrative examples are of different 
size and complexity and yield heterogeneity estimates for 
which the homogeneous variance assumption was violated 
to an extend that impacted the findings of the MTCs. Our 
study is also the first to compare multiple weakly and mod- 
erately informed approaches to modelling heterogeneity in 
MTCs. Our study, however, is by no means generalizable to 
all MTCs. Several treatment networks may exist or emerge 
in which, for example, the homogeneous variance model 
and some heterogeneous variance model will yield close to 
equal inferences about all comparative treatment effects. In 
this vein, it is important that authors and readers of MTCs 
continually pay careful consideration to the fragility of vari- 
ance estimation, credible intervals and treatment rank 
probabilities. Another limitation is the empirical nature of 
this study. With empirical data we can only observe differ- 
ences, but never infer definitively about the truth. In this 
context, simulation studies would be needed to investigate 
the performance of the models based on bias, precision, 
MSE, etc., under different scenarios and types of networks. 
However, we believe additional empirical studies are neces- 
sary to inform which scenarios are truly important to ex- 
plore under simulation. 

Appropriate modelling of heterogeneity variances in 
MTCs will become increasingly important over the next 
years. First, statistical significance' and treatment rank prob- 
abilities can be sensitive to the employed variance structure 
and variance priors [15]. Since regulatory agencies and clin- 
ical decision makers increasingly rely on comparative 
effectiveness inferences from MTCs, choosing the appropri- 
ate variance structures and priors (and necessary sensitivity 
analyses) also becomes increasingly important. 

Further, we will likely see an increase in MTCs incorpor- 
ating meta-regression or subgroup analysis to explain the 
observed heterogeneity by effect modification caused by 
some clinical covariate(s). In this vein, appropriately esti- 
mating the unexplained degree of heterogeneity for each 
treatment comparison is seminal to reliable estimation of 
the effect modification caused by some clinical covariate 
(s). In other words, without unbiased quantification of het- 
erogeneity it becomes increasingly challenging to explain 
heterogeneity. 

Conclusions 

In conclusion, MTC models using either a homogenous 
variance structure or weakly informative variance priors in 



connection with an unrestricted heterogeneous variance 
structure both have serious methodological shortcomings. 
Using informative variance priors in connection with an 
unrestricted variance structure or borrowing strength by 
assuming exchangeability or imposing consistency be- 
tween heterogeneity variances, can all ensure sufficiently 
reliable and realistic heterogeneity estimation, and thus 
reliable MTC inferences. All four approaches should be vi- 
able candidates for replacing or supplementing the conven- 
tional homogeneous variance MTC model, which is currently 
used widely in practice. 
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